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For bandlimited -functions, simidtaneous approximation of a function 
and several of Us derivatives is considered. Concomitant entropy estimates 
are obtained. A feasible algorithm for the transmission of information is 
discussed. This algorithm has been applied to the design of a class of PCM 
systems. 1 

I. INTRODUCTION 

It is the purpose of this paper to discuss both the best approximation 
of sets of bandlimited functions under Sobolev norms and the con- 
comitant information-theoretic estimates. The Sobolev norms are 
useful when it is desired to approximate simultaneously the function 
and some of its derivatives. This requires an amount of information 
beyond that for approximating only the function. Section II gives the 
necessary background definitions of width, entropy, and capacity; 
theorems providing representations of bandlimited functions, as well as 
a form of Mitjagin's inequality relating approximability to entropy, 
are proved. The distinction between capacity and entropy is comparable 
to that between communication and storage, since capacity refers to 
the number of distinguishable functions transmitted from a signal 
source while entropy measures a bit requirement for the reproduction 
of a function to within a specified accuracy. A constructive approach 
to communication requirements implies an explicit means of representing 
any function of the signal source by numbers with a uniformly bounded 
number of digits. The procedure or algorithm used is usually obtained 
from an infinite series representation with subsequent truncation and 
quantization. Pulse code modulation systems provide examples of this 
procedure. Section II gives a precise definition, while Section III presents 
an explicit construction of a feasible algorithm. This algorithm has been 
applied to the design of a class of PCM systems. 1 
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Sections III and IV contain the theorems and proofs which provide 
upper bounds on widths and entropies. Section III discusses signal 
sources with finite instantaneous power. Section IV considers signal 
sources in which the total energy is finite. 

II. PRELIMINARIES 

Let A be a subset of a Banach space X; it is desired to approximate A, 
that is, uniformly all elements of A by means of n-dimensional sub- 
spaces X n of X. The deviation E Xn {A) of A from %• is defined by 

E Xn (A) =sup inf ||/- ||. (1) 

ftA BtXn 

The deviation provides information on how well A may be uniformly 
approximated by elements of the given space X n ; however, another 
choice of X n might provide a smaller deviation. Accordingly the nth 
width, d x n (A), of A relative to the space X is defined by 2 

d x (A) = inf E Xn (A). (2) 

X„<zX 

If the infimum is attained, then a corresponding X n is called an extremal 
subspace. The following properties are immediate. 

^ df +1 (A) ^ d x n (A), n S= 0, (3) 

d x (A) = sup || x ||, (4) 

XtA 

BCA^ d x n (B) ^ d x (A). (5) 

If X has finite dimension m, then d x (A) = for n ^ m. 

A set of sets whose diameters do not exceed 2e(e > 0) and whose 
union contains A is called an e-covering of A. A finite set S C X such 
that for / c A there is a g t S with || / — g \\ ^ c is called an e-net of A. 
Clearly d x (A) ^ e for a set A possessing an e-net of n elements. If A is 
totally bounded then lim„_^, d x (A) = 0. To see this, choose a covering 
of A consisting of n eballs, then their centers constitute an e-net of A. 

Let N t (A) (presumed finite) be the number of sets in a minimal 
e-covering of A; then the absolute e-entropy, H,(A), of A is defined by 

H t (A) = log N t (A) (6) 

in which the logarithm is taken to base two. 

Let N X (A) be the number of elements in a minimal e-net S C X of A; 
then the relative e-entropy, H X (A), is defined by 

H*(A) - ^g N*U) (7) 
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in which the logarithm is taken to base two. 2-4 For A totally bounded, 
let Xi , • • • , x„ be the elements of an e-net, and let 5,(1 ^ j i g£ n) be a 
ball of radius e about r, ; then the sets U,- = Bj C\ A constitute an 
e-covering of A ; hence 

H,(A) ^ H*(A). (8) 

The minimum number of binary digits, d, of an integer expressed in 
radix two needed to identify uniquely every element in a minimal 
e-covering of A satisfies 

[H € (A)) £d£ [H t (A)) + 1 (9) 

in which [x] designates the integral part of x, that is, the unique integer 
satisfying x — 1 < [x] ^ x. Thus H t (A) may serve as an absolute 
measure of efficiency for processes designed for the storage and trans- 
mission of information. 

Let a set w of n real numbers be chosen, and also a mapping from A 
onto ftp = co X • • • X co (p times); that is, 

x z A — » a = (oti , • • • , a v ) £ ftp , ax , • • • , or p £ co. 

Let the algorithm T define a one-to-one and onto mapping of S2 P to an 
e-net S of A in which T(a) t S approximates x t A to within e; then the 
volume V(T) is defined by 

V(T) = p log n. (10) 

In view of expression (8), one has 

V(T) ^ H X ( (A) ^ H t (A). (11) 

Thus the greater V(T) is, the less efficient is the algorithm T compared 
to the absolute standard H ( (A). 
If D C A has the property that 

j^g, 1, gtD=*\\f-g\\ > e, (12) 

then D is called e-distinguishable. Let M t (A) be the number of elements 
(presumed finite) in a maximal e-distinguishable subset of A, then the 
e-capacity, C,(A) is defined by 

C.(A) = logM.(A), (13) 

the logarithm being again taken to base two. 3 For a transmission 
system, C,(A) measures the number of distinguishable signals of the 
source or of the processed signal at the output of the receiver depending 
on the identification of A. The following inequalities hold between 
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e-capacity and e-entropy: 

C 2l (A) ^ H.U) ^ CM)- (I 4 ) 

To show this, consider the inequality on the right. Let D be a maximal 
e-distinguishable subset of A; then e-balls about each element of D 
constitute an e-covering of A for, otherwise, there would be an x t A 
not covered and hence more than e away from every element of D. This 
would contradict the maximality of D. For the inequality on the left, 
let D be a 2edistinguishable subset of A, then the number of elements 
of D cannot exceed the number of covering sets of diameter 2e or less 
in an ecovering of A for, otherwise, there would be at least two elements 
of D in the same covering set. This would contradict the 2edistinguish- 
ability of D. 

It is possible to bound H X (A) above and below in terms of d x (A) 
(refer to Ref. 2 where Mitjagin's inequalities are given). An improved 
form of Mitjagin's upper bound is proved below. 

Theorem 1: Let A be a totally bounded subset oj a real, normed, vector 
space X. Let the nth widths relative to X be d x (A)> an d let 

N = max [n : d x ^(A) ^ (1 - a)e] 
with a an arbitrary number satisfying < a < 1 ; then 

Proof: Let E N be an ^-dimensional subspace of X for which E Bn (A) < 
(1 - a)e, then VxtA3ytE N 3 \\ x - y \\ < (1 - a)e. Let A N be the 
set of all such y for every x e A. An a e net of A N is an e-net of A; hence 
H x t (A) ^ H x t (A N ) ^ C a( (A N ). Letyi , ■ ■ ■ , y u be an ae-distinguishable 
subset of A N , and let B k C En be balls with centers y k and radius \at, 
then they are disjoint and are all contained in the ball B with center 
the origin and radius d x + (1 - ha)e. Let X* be the volume element 
in E N ; then \ N M{\at) N ^ \ N [d x + (1 - §a)e]". The inequality of the 
theorem follows on taking logorithms. 

The class of functions to be studied consists of the space B a defined 
by the conditions that f(t) e B a be analytically continuable into the 
complex plane as an entire function of exponential order one and type a, 
and that it be bounded on the whole real axis — °o < t < oo . The follow- 
ing inequality is valid for B„ : 5 

sup | f(t) | £ <r sup |/(0 |. (15) 

— 00<f<00 — O0<«<00 

Important subspaces of the space B a are the space C„ defined by 



in which 
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1tC.=>A(i) = o(e"") (16) 



,1(7,) = sup |/G + ii{) I, t, r, real, (17) 

-oo<{<« 



and the space W„ denned by 

1*W a =>ftL\-«>, oo). (18) 

Several representations for B„ exist; 6 however, the following repre- 
sentations are needed for the present investigation. Let 

*ft *) = *^t • ( 10 > 

at 

0,(i, <r) = <f>(t - jh, o), h=ir/a, (20) 

then one has the following: 
Theorem 2: 

j(t) E c„ «=> /(/) = £ ftjtyfcft <0 

/or aZZ complex t. The series converges uniformly in every closed, bounded 

region. 

Prooj: Consider the integral 



-— I 

2iri J, 



-dr, r-$ + »i, (2i) 



c* (r - ^n erf 

taken over a square Cjv with corners at (N + £) (±1 ±t)ft, and A'' so 
large that t is in the interior of the region bounded by Cat . The theorem 
is clearly true when t = kh (k integral); it will hence be assumed t ^ kit 
for any integral k. The index N ^ is an integer. Evaluation of I N by 
use of residues yields 

f(t) = E /(#)*,(*, o) + /.v -sin crt; (22) 

, = -.v 

thus, to prove the implication to the right, it is sufficient to show I N — » 0, 
N — > oo . Let /.v 1 ' be the integral (21) extended over that part of C^ given 
by* = (N + \)h; then 

i_ ( .v +J) * | (AT + $)h + it, - M I sin (t(N + J) + iVij) | ^ (23) 
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Since 

I (JV + \)h + in - 1 1 £ (N + i)A - 1 1 1 , 
| sin (*■(# + \) + io-u) I = cosh tt? ^ K'"'. 



one has 



,(/v+i)» 



I r« } I ^ r— I / e- aUl A( v ) d-q. (25) 

Writing equation (25) in the form 

1 In l -T(2N + l)h-2\t\ (2JV + l)/i J-ur*»)» 

(26) 

using equation (16) and the following lemma 7 

Kv) -> o, I >j I -* °° => lim 2r /_ ^ rf7? = 0> (27) 

shows that Itf 1 -* uniformly in t. The same conclusion applies to the 
integral extended over g = — (iV + £)/*. 

Let I^ 2) be the integral (21) extended over v = (N + i)h] then 

1 iA ' - 2tt 

r +in m + m + *w * (28) 

■J-or+ii. | { + f(iV + «A - « | I sin 0* + ^ + *)) I 
Since 

| ^ + z'(A^ H- i)7i — i | ^ (-^ + *)^ — M I. (29) 

| sin (<r£ + it(N + *)) I fc Lz ^ Ll e' (A ' +i) , 
one has 

I '■>" I S ^37^ ^y-aH l'-'^'^ + m - (30) 

In view of equation (16), 1^ -> uniformly in t. The same conclusion 
applies to the integral extended over y = — (# + §)& For the implica- 
tion to the left, one may observe that <f>j(t, a) e C a , and that the series 
converges uniformly. 

The series of Theorem 2, which is clearly interpolatory, is called the 
cardinal series. 8 
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For /(/) e !/"'( — 00,00), the Fourier transform relations are given by 
F(u) - —^ /_" M e-""7(0 dt, (31) 

m = ~&$ F-S' nu) du ' (32) 



The Fourier transform of faty, a) is 

1M» 



*#(«i») = -[2) e " u . I«l <»; 

= 0, I W I > (T. 



(33) 



The Parseval relation now shows that the sequence <£,(£, <r), — « < j < °o 
is orthogonal over (— <x>, °o) with respect to unit weight; thus, 



(34) 



f hit, o)<t> k (t, <r) dt = 0, j ^ k; 

= h, j = k. 

The following theorem may now be stated for / c W„ . 
Theorem 3: / tW a 

=> P I fit) \ 2 dt = h E I /(|fc) I 2 , 
/(<) = (2^1 f_/ ut FV) du, 

m = tAi i: /o'/oe-""', 1 1* 1 < ff . 

Proof: The Paley-Wiener theorem 9 shows that / e W„ has the repre- 
sentation given in Theorem 3; hence, by the Cauchy-Schwartz inequality 



fCS + in) I ^ {^f £ I F(«) I 2 du}* - o(e""). 



(35) 



Equation (35) shows that W a C C a ; thus, by Theorem 2, / is in the 
closure of the system <£,(£, a), — 00 < j < °o. The Parseval relation now 
follows from equation (34). To establish the formula for F(ii), it is 
only necessary to show 

/ e"' E 1(jh)e~ iu,h du-^0, M,N-> » , M , AT _> _ 00 , (36) 
because each term is the Fourier transform of the corresponding 0, term 
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of the cardinal series. One has 

I f e iu ' E K]h)e- iuih du ^2af f f(jh)$- Mk ' du, (37) 

| J -a i-M "'-» I i-' w 

I f «*"' E f(jh)e~ <uih du T £ 4<r a fj | f(jh) | 2 -» 0. (38) 

| J-a i-M I i-M 

The limit zero is obtained as a consequence of the Parseval relation of 
Theorem 3. 
To obtain a representation for the class B a , 10 let 



5(7 



to > integral, < 5 < 1, 
>#W = **-*), /i = -(1 - 5); 



(39) 



(40) 



then one has 
Theorem 4-' 1 e B» 



m = £ /w**w- 



TVie series converges absolutely and uniformly in every closed, bounded 

region. 

Proof: The function 

8a 



Km 



sin 



(1 - 8)m 



(s~ t) 



5a 



(1 - 8)?n 



(s- t) 



(41) 



belongs to W a/0 - S) for each positive integer m and arbitrary s, hence the 
cardinal series applied to this function yields the expansion 



sin 



M 



(1 - 5) to 



(s- 



8a 



(s- t) 



(1 - S)m 

= E /0%)i 



sm 



5a 



(1 - 5)?n 



(a - jh) 



8a 
{ (1 - 8) TO 



(s - jh) 



*yU 



1 - 5 



(42) 
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Let s = t; then the required representation is obtained. The absolute 
convergence follows from the boundedness of | f(jh) | and 

I 9,® | = 0( \j |— l ). (43) 

Approximation will be studied in the uniform norm and the following 
Sobolev norm 



/ II. = \f' 2 (l/W I 2 + mi 1/(0 l 2 + ■•• + #». |/ (,) (0 \*)dt\ 

(.J -T/2 J 



(44) 



in which Hi , ■ • ■ , p t are positive numbers. For the space B a , the symbol 
Bl\. will be used for the corresponding normed space. The symbol 
B T a _ t {M) will be used for the subset defined by | f(t) | ^ M, — oo < t < *>. 
For the space TF„ , the corresponding normed space will be denoted by 
Wf tt , and W* tt (B) for the subset in which 

{/" ' Kt) |2 ^ ~ B - (45) 

III. THEORETICAL INVESTIGATION OF B a 

Let B T a designate the vector space B„ normed by 

N/11. - max |/(i)|, (46) 

- T/2 S l £ T/2 

and let B T „(M) be the subset of B r B satisfying 

|/(0 | % M , -oo < t < oo. (47) 

The completion of B T „ is the space C T of functions continuous over 
[-T/2, T/2] and normed by equation (46). 
Let 

o-7' . /2c\* 2c 



c = ^- , 5 n = 1 - I — , n > - , (48) 

2 \7ra/ 7T 






to ^ 1; 



then the following theorem provides a bound on the nth width, 
#*(B*(If))i of 5^(M) relative to the space C T . 

Theorem 5: <% T (B T .(M) ^ (2ilf/irro)e" m . 

Proof: The series representation of Theorem 4 will be used. The func- 
tion 

g(t) = £ IWOM (49) 

lyis.v 
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establishes an approximation to f(t) whose error is given by 

Kt) - 0(0 = E /<jfc)*iO). 



(50) 



From equations (39) and (40), one has 

1 



I »,<0 I i ~ I™, 

T \ird 



] 



T 
2/i 



T T 

i|>§, IMS f (5D 



Define the function p{x) by 

p(rr) =±-x £ X < 1, 
= pix + 1) for all re, 

then the Sonin (Euler-Maclaurin) summation formula 11 is 



(52) 



£ W(j) = f W(x) dx + P (x)W(x) - [ p(x)W'(x) dx (53) 



in which a < 6 are arbitrary numbers. Use of equation (53) with 

1 



»»-;(a 



re — 



T 



re > r/ft, 



2/w 

a = AT + |, 6 



yields 



E i m i ^ -£- 



irm 



VI 



,»iv + |-| 



Let 



then 



m = 



[?(*+§ 



T 

2/i 



^ i; 



S !*,(*) I ^i!:e- m . 



irm 



Thus, from equation (50), one obtains 



' " " irm 



(54) 



(55) 



(56) 



(57) 



(58) 
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and hence 



For ft odd, one has 



inn 



(59) 



dV{B T .w) ik ™ 



exp 



fe(. 



fe(-S 



(00) 



while if ?i is even, one has d° ^ d£_, ; hence, in all cases 



£' s ^ 



exp 



fe(- 



- 1 - 



if ( n - ! - f 



(01) 



The fractional guardband 5 is now chosen as in equation (48) from which 
the inequality of the theorem follows. 

When n is large, a more accurate estimate of d° T may be obtained by 
using a polynomial approximation to B T a . Let 



1(0 =/(fs) = 9(-r), 



(02) 



and let L(x) be the Lagrange interpolation polynomial established for 
g(x) on the zeros of T n (x), the ?ith Tchebysheff polynomial of first kind, 
over [ — 1, 1]; then the standard error formula for Lagrange interpola- 
tion 11 yields 



max | g(x) - L(x) | ^ . 9 »-i max | g (n) (x) 
Bernstein's inequality (15) and equation (62) now yield 



«<>-4<) *f (I) 



ft! \2/ ' 



(63) 



(04) 



hence one has 
Theorem 6: 



CT{ D T 



d: \b;(M)) ^ 



2.1/ cY 



n > 0. 



Let H T , be the space of functions /(/) possessing derivatives up to 
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order s satisfying /,/,-•-, / (8) e L 2 (-T/2, T/2) and normed by equa- 
tion (44); then Theorem 7 provides an estimate of the nth width of 
B? f .(M) relative to H T , . 

Theorem 7: Let 

„_/?* £ , 2_ U 

" IS T 2r (s - r - l)! 2 (2s - 2r - l)(s - r) "^ T 2 ' M ' 

in wAtc/i /xo = 1 and the sum is considered zero when s = 0, then 

<&'. r (£j..(AO) ^ 



*'/»* ,,™ ^ Mr r l (2c)" + '_ 



Proo/: For the function 0(3) of equation (62), the identity 

g(x) = P(x) + /* ( ^1 tt j)7 ( » rfw > (65) 

(in which P(x) is a polynomial of degree not exceeding s — 1), will be 
used to obtain a polynomial approximation to g(x) in the Sobolev norm 
(44). Let L(x) be the Lagrange interpolation polynomial for g {,) {x) 
formed with n nodal points on [—1, 1] and u(x) the corresponding 
fundamental poynomial; then one has 

g w (x) = L(x) + ~j g^O-Mx), * e [-1, 1]. (66) 

The polynomial I(x) defined by 

7(.r) = P(x) + J* ( fe~ tt l)V L («) rfw ( 67 > 

will be used to approximate 0(2) in the Sobolev norm; its degree does 
not exceed n + s — 1. Let 

\g tn (x)\*M lt |s|£l; (68) 

then, from equation (66), one has 

I g M (p) - I tr \x) I 

<^ff fe = Mr " r " 1 , I «(«) I rf«, ^ r < «, (69) 
_ n! J_, (s — r — 1)! ' 

I £<•>(*) - I M (x) I £«J±i|«<0|. (70) 
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The norm (44) for the interval [ — 1, 1] may be written 

II f - JIG - f I>r I 9 M to - I M &) \ 2 dx, (71) 

J—l r-0 

in which v , ■ • • , v, ^ 0; the v r and \i T are related through the change 
of variable t = (T/2)x. Using equations (69) and (70), one has 

II* -i IB* 



nY 



■ L { s *(£ t -?-i)<. i w(u) ' 4 + •*#} dx - (72) 

Define the function k(u, v) by 

k(u, v) — I (x — u)'~ r ~ 1 (x — y)'~ r_1 dx; 

Jmai (u..) 



then equation (72) may be written: 



9-IW.Z 



The Cauchy-Schwartz inequality shows that 



n! 2 l^ (a - r - l)! 2 (2s - 2r - 1) 

»1 \2 ~1 

(1 — w)*" r-i | oi{u) I dw] + v, \ oj(.t) 2 da; 
Further application of the Cauchy-Schwartz inequality yields 



9-I\\.^ 



n\ 



2.-2r 

+ V. 



.f? 2(s-r - l)! 2 (2s - 2r - l)(s - r) 
A good choice for o>(a;) is 



(73) 



n! 2 l^(s -r- l)! 2 
• / / k(u, v) | u(u)u(v) \dudv + v, / «(a:) 2 dx> • (74) 



, , . ^ (1 - M)*~ r_i (l - v)'~ r_i 

&(*, ») ^ * 2a -2r- 1 5 (75) 

hence, 

\\g- I\\.g --i2- 1 2^ 



(76) 



j a(x) 2 dx. (77) 
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o(x) = 



2n 

n 



Pn(x), 



(78) 



where P»(x) is the nth Legendre polynomial. The coefficient of P„(x) 
in equation (78) makes o>(x) monic. Since 



f P n {xf dx = 



2n + 1 ' 



(79) 



one obtains 
11(7-/1 



< 



2"M„ + . 



n 



2n\ \2n + 



t)' 



* r 2 2 



^ 2(s - r - l)! 2 (2s - 2r - l)(s - r) 
The Bernstein inequality (15) shows that 

M n+ . g Mc n+ '; 



+ v. 



(80) 



(81) 



hence 



M2V 



n\ 



2n \ \2n + 1 



02>- 



f? Q 2(s - r - l)! 2 (2s - 2r - l)(s - r) 



+ *. 



(82) 



Finally the change of variable t = (T/2)x and the replacement of v r by 
the original y. r yield the result of the theorem. 

The results of Theorems 5, 6, and 7, may be translated into estimates 
of entropy by use of the Mitjagin inequality of Theorem 1. The estimates 
so obtained will apply only to the subset of B a (M) for which f(t) is real. 
Doubling the bounds will provide estimates for complex valued f(t). 

Theorem 8: Let < a < 1, (1 - a)e < (2M/ire), j(t) real, 

1 l . m 

m = In 



tt(1 - a)e 



tt(1 - a) 



l + In 



m 

tt(1 - ay 



+ In In 



2M 

7r(l — a)e_ 



then 
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{ 2 + ft ( 1 + g(. + .))')]} log (f + ^) 



Proof: According to Theorems 1 and 5, one must solve the inequality 

2/1/ 

— e- ^ (1 - a)e (83) 

trm 

for the largest integer m; thus 



me m ^ 



2il/ 



tt(1 - a)e 
Consider the function 

F(x) = 8 - x - In x, 8 > 1. 

One has 



(84) 
(85) 
(86) 



n*)=-i--; 

hence, by the mean value theorem, 

/,'(5 _ h) = -In 5 + Ml + |j , 5 - h < £ < 5. (87) 



Let 



then, since /* is positive, 



F(5 - h) = 0; 



< -In « + Ml + 



1 



8 - h/ ' 
< - 5 In 5 + /j(1 + 5 + In 5) - h 2 ; 



thus 



and 



h > 



8 In 8 



1 + 8 + In 5 ' 



5 - fe < 6 



1 + 5 



The inequality 



1 + 8 + In 5 
x + In x ^ 5 



(88) 

(89) 
(90) 

(91) 

(92) 
(93) 
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is thus satisfied by 



*<»r+WfcV (94) 



hence, setting 



5 = In 



2M 



tt(1 - a)e 

and taking cognizance of the integral character of m, one has 

2M 



(95) 



m 



In 



271/ 



1 + In 



tt(1 - a)e 



1 + In — — + In In - 



(96) 



7r(l — a)e 7r(l — a)ej 



provided 



(1 - a)e < 



2M 
ire 



For the computation of d n - l , one has from equation (48) 

[|aUn-2)]. 



m = — 



Hence 



1 - 



Let 



then 



accordingly 



^C 2 (n-2) <m + l, 

2c yy 26 (m + 1) 

t(W — 2)/ J x 



n = 2 + -v; 

7T 



U-y < e -(m + iy, 



rt" 



, ! - ^- (m + 1) 



(97) 

(98) 

(99) 
(100) 

(101) 

(102) 

(103) 
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Thus n satisfies 



n ^ 2 + 



v " + G<-h ' 



)T] i 



(104) 



the theorem now follows from equations (96), (97), (104), and Theorem 1. 
When e is small, a more accurate estimate of entropy may be obtained 
by use of Theorem 6 in place of Theorem 5. Accordingly one has 



Theorem 9: 


Let < a < 1, v = (2M/(l - a)e(Tec) h ), 


■q ^ max (— , e c/2 J , /(/) real; 


then 


H t (Bl'(M)) 




r 


2 1n, + l-*ln^ln,) 


. (2M . 2 - « 


£ 1 -r 


(2 \ 1 

In 1 In u l _l_ 1 _L 


r log I + 




\ ae a 


l 


in I m -q] -f- i -\- , 
\cc 1 2 In 7} _ 





Proof: According to Theorem 6, one may consider 

2M (cV 

— [-) * (1 - «)«. 

Stirling's formula provides the inequality 

n\ > n n e~ n (2irn) i , 
and hence one may consider 



Let 



(27m)' V2n 



re 



2M 



[1 — a)e(TecY 



then equation (107) becomes 



x x+1 " c ^ v 2/ ' 



Consider the function 

F(x) = 5 - (x + a) In x; 



(105) 
(106) 
(107) 

(108) 

(109) 
(110) 
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then, by the mean value theorem 
F(& _ h) = 8 - (8 + a) In 8 + h\\n 8 + 1 + | J - j \j - 3) , 



5 - h < £ < 8. (Ill) 



Let 

then 

Let 
then 

and hence 



x = 8 - k ^ a, F(8 - h) = 0; 
^ 8 - (5 + a) In 5 + h In ( 8 + 1 + y ■ 



h > 



x < 



8 ^ 1/ej 

(3 + a) In 5 - 8 

In 5 + 1 + % 
o 

28 + a — ahi 8 

In 8 + 1 + ^ 
o 



(112) 
(113) 

(114) 

(115) 

(116) 



Thus, in terms of n and 77, one has 



n < 



2 In 77 + I - I In I - In r, 



(117) 



The lower bound on j? in the theorem assures the satisfaction of the 
conditions on x and 8 in equations (112) and (114). Use of Theorem 1 
now provides the inequality of the theorem. 

Theorem 10 provides an entropy estimate deduced from the width 
result of Theorem 7. 

Theorem 10: < a < 1, v = (MT(2c)7(l - a)eM*) 

7, ^ max (^ , e c/2 ) , y = (1 + m.«t 8 + ■ ■ ■ + m.O*, /(<) r*rf; 
then 
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H.(BUM)) 

r 



21n? + i-*ln - 



(I'-)" 



_ ln (I ,n ') + ' + 2lhJ 



(Mil! + 2^, 



Proof: The investigation parallels that of Theorem 9. A difference 
occurs in the estimation of d . The Bernstein inequality of (15) shows 
that 



d ^ MyT h ; 



(118) 



hence the estimate of the theorem. 

The case m = 1 of the representation given in Theorem 4 may be used 
to obtain an explicit e-net for B„(M), and hence to provide a constructive 
algorithm for the transmission of information from such a source. The 
representation for f(t) e B„(M) takes the form 



sin 



8a 



at) = E fw 



1 - 5 



(t — jh) sin 



1-5 



(t - jh) 



1 - 5 



(t - jh) 



1 - 5 



(t - jh) 



ah = tt(1 - 8). (119) 

In order to proceed, it is necessary to estimate the quantity A(8) 
given by 

ha 



-4(5) = sup £ 



— oo<(<oo ,--oo 



1 - 8 



(/ — jh) sin 



1 - 8 



(t - jh) 



8a 



(t - jh) 



, 1-8" ' ' 1-5 
Theorem 11: A(8) ^ 1/5* for < 5 < 1. 
Proof: The Cauchy-Schwartz inequality yields 



(t - jh) 



(120) 



A(5) 2 ^ sup £ 



-KXKOO J--0 



5(T . 

sm l- 8 {t 


- jfc) 


[rh»- 


#) 



sup £ 

— OS < l< 08 ,'--30 



sin 


* r/ 


- jh) 


1- 5 ( * 


[ 1 


°" f/ 


jh) 


- 5 ( ' 



(121) 
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From the Parseval relation of Theorem 3, one has 





a (t 


-jh) 




-8 {t 


a 

I 1 - 


B« 


jh) 



sin 


lV 


- jh) 


1 


^~ 


Jh) 



h J- 



h J- 



sin 




a (f 


-s) 


1 


-8 {t 


[ 1 


(T 




s) 


- 


8 {t 



sin 


i-V 


-8) 


I 1 


*!<«- 


•) 



ds= 1, (122) 



ds = V (123) 





The theorem is established. 
Let 

8= up 10ft) |; 

-O0<|<00 

then a corollary to Theorem 11 is 
Corollary: 

sup | /(0 | ^ -S/5 } . 

-00<«<00 

Proo/: From equations (119) and (120), one has 

sup |/(i) | ^ SA(8). 

-oo<t<co 

The result follows from Theorem 11. 
The function 



(124) 



(125) 



Sill 



8a 



g(t) = £ /(i><) 



1 - 5 



(t — jh) sin 



1 - 8 



(t - jh) 



8a 



(126) 



1 - 5 



(t - jh) 



1 - 5 



it - jh) 



constitutes an approximation to j(t). The error may be assessed by 
application of equation (51) for m = 1, and Sonin's formula (53); thus 

2 2/i/ 



f-*\\-*M» ' 



i- v + i + 2i 



(127) 



For < « < 1, let 



'-kr^fr^+ft^JW-fi^™ 



then direct verification establishes 

\\f-g\U < (1 -«)e. 



(129) 
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It may be observed that for large c, one has 



N ^ 



tt(1 - 8) 2h' 



(130) 



that is, N is approximately the number of nodal points jh in (-T/2, T/2), 
Let 



m m [Mi!] 



and 



L 2ae J 



M) = (fi-M, ■•■ ,ftv(/)); 



(131) 



(132) 



then the set Up is defined to consist of all f(t) generating the same vector 
/3 = /3(/). It will now be shown that the diameter of Up does not exceed 
26. Let /x(0, MO e Up ; then 



I fiW - f 2 (jh) | ^ 
One has 

/.(/) - mo = E (/iOfc) - /i0fc)) 



.1(5) 



(133) 



Sill 



1 - 8 



(t — jh) sin 



1 - 5 



(t - jh) 



8a ,, ... 



1 - 5 



V - jh) 



(134) 



and hence, by equation (129), 

I MO - MO I S E I /.(#) - M#) 



sin j _^ - (i - jh) sin ^ (i - jh) 



8a 



(t - jh) 



(t - jh) 



+ 2(1 - «)e (135) 



l- r v J '"' i - 5 

in which .A/ is chosen as in equation (128). From equation (133), one has 

I MO - MO I S % 



• £ 



111 • — - (^ — #0 sin q ; (f — jh) 



1 - 5 



1 - 8 



8a 
1 - 5 



(I - jh) 



1 - 8 



(t ~ jh) 



+ 2(1 - a)e. (136) 
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Use of equation (120) shows that 

||/i-/.||.£2«. (137) 

The sets Up are centerable with respect to themselves; that is, there 
exists an element g(t) t Up whose distance from any other element of Up 
does not exceed e. Consider the function g(t) denned by 

sm T ^-At-jh)sm-^—(t-jh) 

(138) 
One has 



and 

M - gto 



1(jh) - g(jh) I £ ^ , I i I * *. CW 



sin ; a . (i - j'/i) sin (< - j'ft) 






sin -J2-(«-jfc) sin rS <* - *> 

+ E «» — if- 5 tt"* ; (140) 

hence, by equations (120), (129), and (139) 

11/ -0 II. 3* ( 141 ) 

The required constructive algorithm, r, is thus given by the mapping 
/ — > g in equations (131) and (138). 

Theorem 12: V(T) = (2;V + l)log {[A(8)M/2ae] - [-A(S)M/2at] + 1}, 
in which N is given in equation (128). 

Proof: It is necessary to enumerate the number of distinct g(t) which 
are generated by T(B,(M)). Since 

A(d) 1 M) 1 A(3)M ^ (142) 

2ae 2ae 
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the number of distinct values of #,(/) is 

and hence the number of distinct vectors /3(/) is 



2ae \ " I 2ae , 



+ U • (144) 



The theorem follows from equation (144). 

Corollary 1: V(T) ^ (2N + 1) log ([M/Zaetf] - [-yU/'iaeS 5 ] + 1). 
Proof: Theorem 11. 

Corollary 2. V(Y) ^ (2JV + 1) log (M/«€(5)* + 2). 
Proof: Corollary 1 and the inequalities 



[dy ^ ^ - (u5) 



M 

2ae(8f ' 



[-. 



" '<A + .. 



2ae(8)'A 2ae(8)* 

IV. THEORETICAL INVESTIGATION OF W„ 

Using Theorem 3 for /, g e W a , the Sobolev inner product 

(/, g). = f" (3d + v*fg + ■ ■ • + m/"/ 77 ) dt (i4G) 

J-T/2 

takes the form 

T 
sin - (« - y) 



fc *>• = L L -sr^ 



•(1 + MiW + • • • + n s uV)F(u)G(v) du dv, (147) 

in which F(w), G'(w) are the Fourier transforms of/, respectively. The 
corresponding positive definite quadratic form Q is 

T 
sin jr- (u - v) 



«=ii^ii' = L/..-fc 



-(u - y) 

■(1 + /iiwy + ■ ■ • + fjL,u'v')F(u)F(v) du dv, (148) 
and an operator K generating Q is given by 
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T 

, sin - (u - v) 
KF = [ —j r- 

•(1 + MiW + • • ■ + n,u'v')F(v) dv, \u\ ^ a; (149) 
thus 

Q = f FKF du. (150) 

The equation defining the eigenvalues and eigenfunctions of K is 

K$ k = X,* fc , k ^ 0, (151) 

in which the ordering X ^ \i ^ X 2 ^ • • ■ is used. It follows from the 
Hilbert-Schmidt theory 12 that the eigenvalues are denumerable and of 
finite multiplicity and the eigenfunctions form an orthonormal set 
which, from the positive definite character of K, is complete in L 2 (—o, a). 
Let 

tpM = ^i \\ e iul *t(u) du; (152) 

then the Parseval relation for Fourier transforms shows that the 
sequence <j> (t), <f>i(t), 4> 2 (t), ••■ is orthonormal over (- «>, <»); further, 
from equations (147), (150), and (151), one has 

(</>,- , (f) k ). = J <hK$i du = A,- J $,$* du = j t± k 

= X,. j = k. (153) 

Thus the sequence {0*(O1 o forms an orthogonal system with respect to 
the Sobolev inner product (146). The system {<f>k(t)}t is also complete 
in TFJ",. as a consequence of the completeness of the system {$*(w)} o m 
L 2 ( — a, a). 

Define the n-dimensional subspace X n C W t by 

X n = X n (<f> , ■■■ , 4>»-i) (154) 

then Theorem 13 provides the nth width of TTJ*„(5), relative to H T , , 
in terms of the eigenvalues of K. 

Ttieorem IS: ^'(^(B)) = BXj. 

Proof: Let f(t) tW B ,.(B); then 

/(*) - LflAC). (155) 
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Let 

g(t) = 2 ««**(0 * *« ; ( 156 > 

4-0 

then the orthogonality of the <f> k (t), (153), yields 

11/- a IB- EKI 2 **- (157) 

Thus 

inf ||/- all! ^ El«* rx*- (158) 

BtX» t-n 

From the monotonicity of the \ k , one has 

inf ||/-alC2S>.EKf; (159) 

BtZn *~» 

however, the orthonormality of the <£*(£) over (— °°, oo) shows that 

r i /(o i 2 ^ - e i «* i 2 ^ # 2 > (leo) 

and hence from equation (159) 

E Xn {Wl,(B)) = sup inf || / - p ||. £ BK*. (161) 

Thus 

dSWUR) ^ BKf- (162) 

Consider the ball f/ n+1 defined by 

g(t) - Sfl^W, II ? II. £**.*; (163) 

* = 

then, by a theorem on balls in a finite dimensional subspace of a Banach 
space, 2 

d n (U n+l ) = BX„ J . (164) 

Thus the theorem will be established if it is shown that the ball U n+1 
defined in equation (163) is contained in W a ..(B). It is only necessary 
to verify that 

f I g(t) \ 2 dt= 2 I a k | 2 ^ B 2 . (165) 



193fi THE BELL SYSTEM TECHNICAL JOURNAL, OCTOBER 1970 

One has from 

II g II? = EKI 2 a* ^BX (166) 

that 

E|a*| 2 ^ t,\«>\ 9 T£B*> ( 167 ) 

and hence the theorem is proved. 

Use of the series representation of Theorem 4 permits one to estimate 
d"' T (W^ (B)). The quantities m and 5„ are as in equation (48); addition- 
ally, the corresponding interval h n is defined by h n = t(1 — 8 n )/<r. 

Theorem 14: 



d "' <™» ^ =C?7 7 t 

( (2m + l)(n - 1 



Proof: Let 



and 



/(0 = JljWe^t), (168) 

0(0 - E /W*,(0; (169) 

i /law 



then 



1/(0- 0(0 I ^ £ l/0'/0 I I 0.(0 |. (no) 

in>w 

Since, by Parseval's relation of Theorem 3 

f | f(t) | 2 dt = h £ I /(ft | 2 ^ B 2 , (171) 

«/ — oo y = — oo 

Schwartz's inequality applied to equation (170) yields 



/(0 - 0(0 I' as t £ »*«>•■ 

1 l»l>W 



(172) 



One has, from equation (51) 

2B 2 (mY v 





1 


T 

N> 2h- 


(173) 


(,. 


m\2m+2 ' 

" 2h) 
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One may use Sonin's formula, equation (53), to effect the summation 
in equation (173); thus 



11/ all - B ^ 


m 




m 


{h(2m + 1){n + - - £)) 
The choice 


t«(# + | - 


2hJ, 
(174) 


leads to 

II f ill - B v5e_m 
II / Q \\v ik i I 

[h(2m + 1)(aT + | - 




(175) 


2h/J 


(176) 



Thus equation (176) shows that 

- f-V 

7T W / (XT 



dffiUwUB)) £ ^ (f) V -t^-. — rr,; (177) 



( (2m + 1} ^ + I _ |) 

and, hence, for n odd 

((2m + !)(» - I)) 

For ?i even, one has 

( (2m + d( b _!_!)) 

thus equation (179) applies in all cases. The fractional guardband is 
now chosen as in equation (48), and the inequality of the theorem 
follows. 

Theorem 13 permits an immediate corollary to be obtained from 
Theorem 14. 

Corollary: For s = 0, one has 

4 e~ 2m 



X„ ^ — 



(2m + 1} ( n _ 1 _ __ 
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As was done in Theorem 7, polynomial approximation will be used 
to estimate d" T - (J¥* a (Bj). The estimate is given in Theorem 15. 

Theorem 15: 

]H r. nxrT ,™ . n j2c\* (2<T* 



dnl:{W T a . s {B)) ^ BT[~ 



* ni ( 2 r) ((2n + i)(2n + 2s + i})i 



Proof: The estimate will be obtained from equation (80). In order to 
estimate M n+ , , consider 

M = T^Yi f ' eiU ' F( V) du ' (180) 

from which one has 

9to=^f_/ UtT/2) 'nu)du. 081) 

Accordingly 

g{T)(x) = (I)' && L '"""'ww du - (182) 

By use of the Schwartz inequality, one obtains 

[ 9 iT \x) | 2 =£ (1)^ £ /' w 2r du f | F(u) | 2 du. (183) 

The Parseval relation for Fourier transforms 

r | /(<) | 2 dt= f \ F(u) | 2 dw ^ £ 2 (184) 

«.' qq v —a 

and equation (184) now yields 

I fl (r) (Y) I 2 < B 2 -—^- — 
10 W I = J3 T 2r+l 

Thus 



(185) 



*~.*^J p» + » + !)* ■ (186) 

The remainder of the analysis is the same as in Theorem 7. 

Theorem 13 again permits an immediate corollary to be obtained 
from Theorem 15. 
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Corollary: 



T 



(2c) 8 " 



in 



nV I " (2n + l)(2n + 2s 4- 1) 



Theorems 14 and 15 lead to corresponding estimates of entropy 
through use of Theorem 1. 



Theorem 16: Let 



n ^ 2 + 1 + 



2c 



and 





m = 


then 




HXWUB)) 




^ {2 + 





= In 



< a < 1, /(0 reaZ, 



25 /2c 



7rV3~ (1 - a)e \tt 



1 ; (;(« + i)) l ) i ]}iog^= : "" 



(2B\ > , 2 - a 



Proof: From Theorem 14, one has 



rf,.. I (lC„(fl)) rg -B 



2 J 1 



7T \h„-o/ (IT 

'{2m + l)(w -2-r^— 



(187) 



From equation (48), one has 

2m + 1^3, 

hence 



_T_ (2c 
ft.-. = 



*-><rt*M) S faffi,- 



(n-2) ; 



1 



2 ^ 



(188) 



(189) 



Since 



(«• - * - »7 



— , ^ 1 for n ^ 2 + 1 1 + 



2c 



J\ 2 



(190) 
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f/ n _,(TFj n (5)) obeys the inequality 
According to Theorem 1, one may consider 



' *&)'.-. 



(191) 



and hence 



m 



3t*ffi< 



= [ hl C^(f-a)e \f) )_ 



(192) 



(193) 



The remaining analysis is the same as that of Theorem 8. The inequality 
of the theorem now follows. 

Theorem 17: Let < a < 1, v = TB(2c) B /(l - a)ee{-wc)\ 



n ^ max (-0-2 , e c/2 j , j(t) real, 



then 
H t (WUB)) 



< i 



s + 1 + 



21nij + l-ln(-lnij 



In (- In 7j ) + 1+ t^ 

\ec I In n _, 



log \^ 



(lB\} . 2 - a 



Proof: The proof parallels that of Theorem 9. 
It may be useful to observe 



Ao ^ — 7 • 

IT 



(194) 
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